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The time evolution of complex systems usually can be described through stochastic processes. 
These processes are measured at finite resolution, what necessarily reduces them to finite sequences 
of real numbers. In order to relate these data sets to realizations of the original stochastic processes 
(to any functions, indeed) it is obligatory to choose an interpolation space (for example, the space 
of band-limited functions). Clearly, this choice is crucial if the intent is to approximate optimally 
the original processes inside the interval of measurement. Here we argue that discrete wavelets 
are suitable to this end. The wavelet approximations of stochastic processes allow us to define an 
entropy measure for the order-disorder balance of evolution regimes of complex systems, where order 
is understood as confinement of energy in simple local modes. We calculate exact results for the 
fractional Brownian motion (fBm), with application to Kolmogorov K41 theory for fully developed 
turbulence. 
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Many physical systems investigated at present have a 
complex evolution in time. Frequently, the major infor- 
mation we can obtain on their dynamics comes from time 
series of noisy appearance [l]. Q. y ■ Q. 01 ■ These series are 
samples at finite resolution of subjacent stochastic pro- 
cesses, whose properties are better investigated through a 
multi-resolution approach, because the realizations of the 
mentioned stochastic processes are singular everywhere 
functions. This is, for example, the case of 1/ f a noises 
from self-organized systems 
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Thus, singu- 
larities are common, and should be interpreted as details 
that influence function variation at all scales 0, H^ |. 
In this paper we propose a method for entropy analy- 
sis of arbitrary complex time series that takes advan- 
tage of this theoretical standpoint. Moreover, we account 
for the fact that measurements are made at finite reso- 
lution, considering the consequences of sampling, what 
is not fully accomplished in previous approaches. In 
general, other formulations are strongly influenced by 
information-theoretical arguments, applied to the anal- 
ysis of chaotic behavior nj|- We see difficulties in two 
main aspects. First, the notion of complexity that is em- 
ployed in these formulations is based on entropy rates, 
like the Kolmogorov-Sinai entropy, that measure the de- 
gree in which information on initial conditions is lost 
when the systems evolve. This gives a scale of complexity 
ranging from zero (non-chaotic deterministic case) to infi- 
nite (stochastic case). In this scale, finite values quantify 
the complexity of deterministic chaos. As a consequence, 
the problem of defining a proper complexity estimator for 
stochastic processes is substituted by the statistical inves- 
tigation of chaotic deterministic dynamical systems (llj . 
Second, these formulations do not consider the relation- 
ship that scaling has with disorder Our approach 
is tailored to face these difficulties. Here, we begin by 
assuming that the stochastic processes are supported on 



the real axis, and measured at a discrete set of points with 
a sampling interval r, during a time T. The resolution 
of the measurement is N — T/t, and for convenience we 
make T = 1, and r = 1/N = 2~ J (we will work in these 
units). Such measurement results in a loss of informa- 
tion, which depends on two factors: The resolution, and 
the interpolation space in which the stochastic processes 
will be projected The idea is that the mere sampled 
values say almost nothing about a function. Much more 
information is conveyed through the hypothesis on how 
the function varies between the sampled values. This reg- 
ularity hypothesis, usually implicit when we draw smooth 
curves between data points, is an essential element of the 
theory. Without this hypothesis, no information found 
on the discrete and finite data sets can be attributed to 
the subjacent model, that is assumed to hold on the con- 
tinuous support. Now, in the time-scale (time-frequency) 
domain, it makes sense to search a representation i) that 
is minimally affected by the created end singularities, ii) 
that provides an interpolation based on multi-scale ap- 
proximation of the actual singularities, and Hi) that deals 
with the resolution N as a direct experimental parame- 
ter. The first and second requirements are the most cru- 
cial, and establish the way in which the energy (L 2 -norm 
squared) is assumed to be distributed inside the interval 
of measurement, so that this distribution corresponds to 
the singularity structure of the process, seen at a finite 
resolution. As a supplementary condition, the best is 
that the algorithmic complexity grows only linearly with 
the length of the time series. These requirements are met 
if we project the stochastic processes in a discrete wavelet 
space. 



Wavelets are associated with time-scale representa- 
tions. Let \F) be a vector in a Hilbert space. There 
are bases {\ijjjn)} such that \F) can be expanded as 
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FIG. 1: The Haar wavelet decomposition of a stochastic pro- 
cess sampled at the finite resolution AT = 2 15 . The details are 
presented from scale the 2~ 14 to the scale 2 -6 . Note how the 
energy accumulates at the small scale details. This causes 
the high entropy S = 12.52 as compared to the maximum 
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the form of the basis functions being ipj n (t) — 
-^=V ^ *~ 2 2 /" ^ . These functions are called discrete 
wavelets, and are square integrable with zero mean, and 
may have p vanishing moments. The discrete wavelet 
transform of F to this basis is written Dj n F = (ipj n \F) , 
and gives information on the behavior of the function F 
at scale 2 J and time 2 J 'n [l2^ . 

Confining the function in expression Q to a finite ob- 
servation window, and measuring at finite resolution, we 
obtain the approximation 



I/./) = (01/) 10) 
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E <Vy»l/> IVy»> , (2) 



where / is the restriction of F to [0,1]. The function cf) 
is called the scaling function, having the properties of a 
smoothing kernel [12]. We are interested in the fluctua- 
tions, so, without any loss, we employ in our analysis the 
transform / — > f —(f), getting from / a version with null 
time-average. At this point we are ready to express an 
energy and an entropy for the time series, that are also 
resolution dependent approximations of the energy den- 
sity and entropy density for the ideal stochastic process. 
The energy of fj is defined by the ensemble average of 
the L 2 -norm squared, 



E(J) = (fj\fj), (3) 
that is directly determined with the simple formula 
E (-0 = Y?i=i ffi where /, are the sampled values of 



/. The entropy has the definition 

i=o 2- j -i 

S(J) = - E p Jntoga p jn-P*logaP*, (4) 

j=-J+l n=0 



with P jn = \D jn F\ 2 /E{J), and P = | (<f>\f) \ 2 /E{J). 
This expression has similarities with the basis entropy 
cost, using the language of reference 0|. In the limit 
of a single pulse localized around t and with duration 
St ~ kr, with k a natural number, only very few wavelet 
coefficients around the scale kr and localized near t 
will be appreciably different from zero. This charac- 
terizes low entropy processes. On the other hand, the 
white noises (WN) in which the energy is, in average, 
equally distributed on all wavelets (the same power dissi- 
pation at all times and scales) have the maximum entropy 
Smax = J ■ This entropy functional is useful to quantify 
the balance between order and disorder (complexity) for 
different regimes of evolution of complex systems. Ac- 
tually, we expect that the level of disorder in a process 
mainly depends on the energy distributed into the de- 
tails. Consider, for example, the interaction between two 
surfaces that slip one over the other. When there is no 
friction we expect no sound. But if we introduce rough- 
ness, the energy will be dissipated in microscopic colli- 
sions, originating a sound that is classified as a noise, 
due to the excess of details. Indeed, the intervals be- 
tween such collisions are so small that no measurement 
will capture them. The process will continue a noise for 
'high' resolution instruments, like it is for our 'low' reso- 
lution ears. In FIG. we see the wavelet decomposition 
of the sound caused by a piece of plastic pressed and 
pushed on a rough table. The calculated entropy was 
5 = 12.52 for a maximum of S max = 15. The random 
small scale details have amplitudes comparable to that 
of the function during the whole observation, constitut- 
ing important components. The great amount of energy 
distributed on the multitude of details causes this high 
entropy. This is why we suggest that the proposed en- 
tropy is physically appropriate. 

A paradigmatic model for the self-affine processes is 
the fractional Brownian motion (fBm), introduced by 
Mandelbrot and Van Ness in the realm of stochastic 
processes with 1/ /" spectra The fBm is a mono- 

parametric stochastic process Bh with null ensemble av- 
erage, Gaussian increments, and whose correlation func- 
tion has the general form: 



B H {t)B H {s) = ^f{\t\ 2H + \s\ 



-\t — s\ 



(5) 



with crfj ee Bjj (1). The parameter of the process is the 
Hurst exponent H £ (0, 1), which controls the nature of 
the correla tion s. The variance of Bh depends on time ac- 
cording to B H = <j h \t\ , that reduces to the Brownian 
motion form when H = 1/2, representing the uncorre- 
cted case. In this process, for H < 1/2 the increments 
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FIG. 2: The solid line is the theoretical ffim entropy at res- 
olution N = 2048. We see that the SRA algorithm gives a 
very good approximation to the theoretical prediction. Two 
lines represent the effect of Gaussian white noise superposi- 
tion. The (visually) straight dotted line at the upper part 
gives the entropy of simulated pure Gaussian white noise. 



In the Haar wavelet basis one finds exact and simple 
expressions for the resolution dependent energy and en- 
tropy densities of fBm. The Haar wavelets are periods of 
square waves written as 



h jn (t) 



-±=, 2 i n<t<li (n + 1/2) 
-^y, 2*'(n+l/2)<t<2*(n+l) 



(7) 



For Haar wavelets one proves, using the expression 
that 



l a ?, I 2 (l 2 2H ) (2H+1)j 

|A ^- J| = (2H+l)(2H + 2) 2 ' (8) 



where Aj n f = (hj n \f), and bn indicates restriction of 
Bh to [0,1]. This expression gives the scale dependent 
power spectrum in the Haar basis, from which one can 
calculate the energy density of Bh 



present infinite range anti-correlations, while for H > 1/2 
there are infinite range correlations. The Fourier spec- 
trum of fBm follows the power law 



S(f) 



1 



F2H+1 ' 



(6) 



E(H, J) = (b H ,j\b H ,j) 



°\ (1 - 2- 2gJ ) 
(2H + 1) (2H + 2) ' 



(9) 



depending on <jh, that must be determined experimen- 
tally . The entropy density has the final form 



S (H, J) = 2~ 2H (2H + 1) (i_ 2 - 2ff ) [l-2- 2HJ ] 

r 



■log 2 C(ff, J) 



(10) 



-2H\ 



where C (H, J) = [l - 2- 2HJ ] / (l - 2" 

This entropy density is universal and exact for the fBm 
processes sampled without rounding error to a given fi- 
nite resolution. It is very useful since for a great num- 
ber of practical applications there are just low resolu- 
tion data, what makes mandatory that the theoretical 
predictions take resolution into account explicitly. In 
the limit J — > oo the entropy diverges at H = as 
S(H, oo) - l/2i71n2. For other values of H the en- 
tropy is finite at infinite resolution. At finite resolution 
5(Q,J) = (J-l)/2 + log 2 (J). 

The FIG. El shows a plot of fBm theoretical entropy 
for the resolution N = 2048. The theoretical result is 
compared with S (H, J) calculated from samples simu- 
lated with the Successive Random Additions (SRA) algo- 
rithm [3| • It shows also how Gaussian white noise (WN) 
affects the entropy, clarifying what is to be expected from 
noisy data. For comparison, the dots at the upper part of 
the figure show the entropy for the simulated pure Gaus- 
sian white noise, whose average value is S — 10.98 ~ 11. 



In the dotted line we see the effect of a superposed noise 
with 10% of the amplitude of the fBm. The dash-dotted 
line shows this effect when the percentage grows to 50%. 
The entropies for all simulated processes were calculated 
from 100 samples. The drastic diminution of entropy 
when H increases happens because the details (rough- 
ness) become weaker. There is very good agreement be- 
tween theoretical prediction and simulation. 

In conclusion, we propose that the entropy measure <@J 
is suitable to estimate the complexity of time series by 
revealing the degree in which detail modes are excited 
in a process. These detail modes are easily represented 
as localized oscillations or wavelets, which in turn are 
very effective mathematical instruments for time-scale 
analysis. Considering such phenomena as fluid or scalar 
turbulence [lfil l2fT |. that are characterized by fluctuat- 
ing cascades from high inertial scales to low dissipative 
scales, or plasma turbulence in which some scales 
are most important for the energy transfer, the pro- 
posed entropy seems a natural measure of the intrinsic 



4 



order-disorder balance. In such phenomena, many struc- 
tures (vortices, convective cells) appear at intermediate 
scales, that should be detected as excitations of inter- 
mediate scale modes, lowering the entropy, as compared 
with noises that are completely determined by micro- 
scopic dissipation (whose characteristics are indicated in 
FIG.^J. For example, if a theory with the same corre- 
lations of Kolmogorov K41 theory is valid, implying 
H = 1/3, the time series of velocity increments would 
lead to the entropy 5(1/3, oo) = 4.27. . ., a number sig- 
nificantly small, compared to the case in FIG. What 
we learn from the ffim analysis presented here suggests 
that the measurement of the entropy is a strong method 
of characterization. Furthermore, it is interesting to ob- 
tain experimental curves of entropy for stochastic pro- 
cesses depending on parameters, or in situations where 
the entropy may change with time. This method has 
given results (which will be published soon) in three sit- 
uations: In the analysis of the sound of lungs, in the 
analysis of heart beat series, and in the analysis of global 
positioning system errors caused by equatorial spread F 
in ionospheric plasma. We observe that such entropy 
measurements could give important information on the 
non-equilibrium approach to self-organized regimes [2fl| . 
as well as, on self-organized regimes made unstable, for 
example, by diffusion [iij . 
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